chore: centralize URL resolution, subdirectory routing, and deployment fixes (PR #243 fixes)#244
chore: centralize URL resolution, subdirectory routing, and deployment fixes (PR #243 fixes)#244sheepdestroyer wants to merge 20 commits into
Conversation
…tions in AGENTS.md
…vironment variable
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
… single x570.vendeuvre.lan domain
…leak / TLS bypass
…up false failures
…al_host/port resolution, fix agy startup SSH commands
…uccess endpoints, nohup stdin redirect)
…add fallbacks, avoid duplicate variables)
…PLACEHOLDER, HOME_PLACEHOLDER, and RUN_USER_PLACEHOLDER
…g subpath if missing
…tilizing PUBLIC_BASE_URL
Reviewer's GuideCentralizes public/base URL resolution for the router and dashboards, introduces environment-driven domain/subdirectory routing, fixes deployment and health probe configuration, and restores portable placeholders in infra templates. Sequence diagram for centralized dashboard URL resolutionsequenceDiagram
actor User
participant RouterApp as Router_app
participant resolveExternal as resolve_external_urls
User->>RouterApp: GET /dashboard
RouterApp->>resolveExternal: resolve_external_urls(request)
resolveExternal-->>RouterApp: langfuse_url, litellm_url, llama_url
RouterApp-->>User: HTML dashboard (links use langfuse_url, litellm_url, llama_url)
File-Level Changes
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
📝 WalkthroughWalkthroughThis PR introduces configurable public base URL support (ROUTING_DOMAIN/PUBLIC_BASE_URL) across start-stack.sh, pod.yaml, and router/main.py's dashboard, replaces hardcoded developer host paths with deployment placeholders, updates litellm model/fallback configuration, refreshes the free-models roster, and updates deployment/README documentation. ChangesPublic base URL and deployment placeholder changes
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant Client
participant Dashboard as "router/main.py (/dashboard)"
participant resolve_external_urls
participant Env as "PUBLIC_BASE_URL/ROUTING_DOMAIN"
Client->>Dashboard: GET /dashboard
Dashboard->>resolve_external_urls: request
resolve_external_urls->>Env: read PUBLIC_BASE_URL/ROUTING_DOMAIN
alt public base URL configured and valid
resolve_external_urls-->>Dashboard: langfuse_url, litellm_url, llama_url (public)
else fallback to local
resolve_external_urls-->>Dashboard: langfuse_url, litellm_url, llama_url (local)
end
Dashboard-->>Client: rendered dashboard with resolved links
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Hey - I've found 1 issue, and left some high level feedback:
- In
resolve_external_urls, the local-development fallback hardcodes:3001,:4000/ui, and:8080; consider deriving these from the existingLITELLM_URL,LLAMA_SERVER_URL, and Langfuse configuration constants to avoid future port/config drift. - The default
ROUTING_DOMAINinresolve_external_urlsand thePUBLIC_BASE_URLdefault instart-stack.shboth encodex570.vendeuvre.lan/vendeuvre.lan; consolidating these into a single shared configuration source would reduce the chance of subtle misalignment across components. - The new production deployment checklist in
.agents/AGENTS.mdhardcodes theboyuser and specific host paths; consider parameterizing these (e.g., with placeholders or variables) so that the instructions are less tied to one machine/user and easier to adapt safely.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- In `resolve_external_urls`, the local-development fallback hardcodes `:3001`, `:4000/ui`, and `:8080`; consider deriving these from the existing `LITELLM_URL`, `LLAMA_SERVER_URL`, and Langfuse configuration constants to avoid future port/config drift.
- The default `ROUTING_DOMAIN` in `resolve_external_urls` and the `PUBLIC_BASE_URL` default in `start-stack.sh` both encode `x570.vendeuvre.lan`/`vendeuvre.lan`; consolidating these into a single shared configuration source would reduce the chance of subtle misalignment across components.
- The new production deployment checklist in `.agents/AGENTS.md` hardcodes the `boy` user and specific host paths; consider parameterizing these (e.g., with placeholders or variables) so that the instructions are less tied to one machine/user and easier to adapt safely.
## Individual Comments
### Comment 1
<location path="router/main.py" line_range="3093-3094" />
<code_context>
+ domain = os.getenv("ROUTING_DOMAIN") or "vendeuvre.lan"
+ if not isinstance(external_host, str) or not re.match(r"^[a-zA-Z0-9.-]+$", external_host):
+ external_host = "localhost"
+ if not isinstance(external_netloc, str) or not re.match(r"^[a-zA-Z0-9.-]+(?::\d+)?$", external_netloc):
+ external_netloc = "localhost"
+
+ # Enforce strict domain validation to prevent loose substring match bypasses (e.g., attacker-vendeuvre.lan)
</code_context>
<issue_to_address>
**suggestion:** The netloc validation regex excludes valid forms like IPv6 literals and hostnames with non-standard characters, which may be too strict.
The `external_netloc` regex only accepts `[a-zA-Z0-9.-]` plus optional `:port`, which excludes valid cases like IPv6 literals (`[::1]:4000`), IDN/punycode hostnames, or reverse-proxy-modified netlocs. In those cases, silently forcing `localhost` can mask misconfigurations and change behavior unexpectedly. If the intent is mainly to filter clearly bad inputs, consider either relaxing validation (e.g., use `urlparse` and trust its hostname parsing) or explicitly logging/rejecting invalid values instead of defaulting to `localhost`.
Suggested implementation:
```python
import os
import re
import logging
from urllib.parse import urlparse
logger = logging.getLogger(__name__)
```
```python
external_netloc = parsed.netloc or "localhost"
external_scheme = parsed.scheme if parsed.scheme in ("http", "https") else "https"
else:
external_host = request.base_url.hostname or "localhost"
external_netloc = request.base_url.netloc or "localhost"
external_scheme = request.url.scheme if request.url.scheme in ("http", "https") else "https"
domain = os.getenv("ROUTING_DOMAIN") or "vendeuvre.lan"
# Basic sanity-check on external_host, but don't over-restrict valid hostnames;
# fall back to the request base URL rather than silently forcing localhost.
if not isinstance(external_host, str) or not re.match(r"^[a-zA-Z0-9.-]+$", external_host):
logger.warning(
"Unexpected external_host %r, falling back to request.base_url.hostname (%r)",
external_host,
request.base_url.hostname,
)
external_host = request.base_url.hostname or "localhost"
# Relax external_netloc validation: use urlparse so IPv6 literals, IDN/punycode,
# and reverse-proxy-modified netlocs are supported. Log and fall back instead of
# silently forcing localhost when invalid.
if isinstance(external_netloc, str):
parsed_netloc = urlparse(f"{external_scheme}://{external_netloc}")
if not parsed_netloc.hostname:
logger.warning(
"Invalid external_netloc %r, falling back to request.base_url.netloc (%r)",
external_netloc,
request.base_url.netloc,
)
external_netloc = request.base_url.netloc or "localhost"
else:
logger.warning(
"Non-string external_netloc %r, falling back to request.base_url.netloc (%r)",
external_netloc,
request.base_url.netloc,
)
external_netloc = request.base_url.netloc or "localhost"
```
- Ensure that `router/main.py` does not already define a `logger` with a different convention. If it does, reuse the existing logger instead of creating a new one.
- If your codebase uses a centralized logging configuration or helper (e.g., `from .logging import logger`), adapt the `logger` usage accordingly rather than importing `logging` and creating a new logger in this file.
- If the actual import section in `router/main.py` does not exactly match `import os` / `import re`, adjust the first SEARCH/REPLACE block to fit the existing imports while still adding `import logging` and `from urllib.parse import urlparse` plus the `logger = logging.getLogger(__name__)` line.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
There was a problem hiding this comment.
Code Review
This pull request introduces dynamic URL resolution for the dashboard and external services (Langfuse, LiteLLM, Llama.cpp) by replacing hardcoded local paths with configurable placeholders and environment variables across pod.yaml, start-stack.sh, and README.md. It also re-enables the local Qwen model, updates the free models roster, and adds deployment documentation. Feedback is provided to ensure that external_host and external_netloc are kept in sync during validation to prevent mismatched URL generation.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| if not isinstance(external_host, str) or not re.match(r"^[a-zA-Z0-9.-]+$", external_host): | ||
| external_host = "localhost" | ||
| if not isinstance(external_netloc, str) or not re.match(r"^[a-zA-Z0-9.-]+(?::\d+)?$", external_netloc): | ||
| external_netloc = "localhost" |
There was a problem hiding this comment.
If either external_host or external_netloc fails validation, they should both be reset to 'localhost' to keep them in sync. Otherwise, if external_netloc is reset to 'localhost' but external_host remains valid (e.g., matching the domain), is_valid_external will evaluate to True and construct a mismatched URL using 'localhost' as the netloc.
| if not isinstance(external_host, str) or not re.match(r"^[a-zA-Z0-9.-]+$", external_host): | |
| external_host = "localhost" | |
| if not isinstance(external_netloc, str) or not re.match(r"^[a-zA-Z0-9.-]+(?::\d+)?$", external_netloc): | |
| external_netloc = "localhost" | |
| if not isinstance(external_host, str) or not re.match(r"^[a-zA-Z0-9.-]+$", external_host) or not isinstance(external_netloc, str) or not re.match(r"^[a-zA-Z0-9.-]+(?::\d+)?$", external_netloc): | |
| external_host = "localhost" | |
| external_netloc = "localhost" |
…ack URLs, and generalize AGENTS.md runbook
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
litellm/config.yaml (1)
9-9: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winStale comment contradicts the re-enabled model.
Line 9 states
local-qwen-3.6 (35B) disabled 2026-06-08but the model is now re-enabled in every fallback chain (lines 41–63) and in themodel_list(line 93). Update or remove this comment to avoid confusion.📝 Proposed fix
# ------------------------------------------------------------------------- - # local-qwen-3.6 (35B) disabled 2026-06-08 — frees 23GB RAM/GTT. + # local-qwen-3.6 (35B) re-enabled 2026-07-08 — llama.cpp on port 8081. # -------------------------------------------------------------------------🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@litellm/config.yaml` at line 9, The comment for local-qwen-3.6 (35B) is stale and contradicts the current configuration. Update or remove the note near the top of config.yaml so it reflects that the model is re-enabled, and make sure the wording matches the active fallback chains and model_list entries in the same file.
🧹 Nitpick comments (2)
litellm/config.yaml (1)
93-99: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick winHardcoded
api_keyin version-controlled config.
api_key: local-tokenis a plaintext credential in a tracked config file. If this is a dummy token for a local llama.cpp instance, consider documenting that explicitly or sourcing it from an environment variable to avoid setting a precedent for committing real secrets here.🔐 Suggested approach
- litellm_params: api_base: http://127.0.0.1:8081/v1 - api_key: local-token + api_key: os.environ/LOCAL_QWEN_API_KEY model: openai/local-qwen-3.6 request_timeout: 300🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@litellm/config.yaml` around lines 93 - 99, The config entry under the local-qwen-3.6 model in litellm_params uses a hardcoded api_key, so update that block to avoid committing a plaintext credential. Replace the literal token with an environment-sourced value or a clearly documented placeholder for the local llama.cpp setup, and keep the change localized to the existing local-qwen-3.6 configuration.router/main.py (1)
21-29: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueDuplicate
urlparseimport.
from urllib.parse import urlparseappears on both line 21 and line 25. The second import is redundant.♻️ Remove the duplicate import
from urllib.parse import urlparse from circuit_breaker import get_breaker from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator, RootModel from typing import Dict, Optional, Union -from urllib.parse import urlparse LITELLM_URL = (os.getenv("LITELLM_ADMIN_URL") or "http://127.0.0.1:4000").rstrip("/") LLAMA_SERVER_URL = (os.getenv("LLAMA_SERVER_URL") or "http://127.0.0.1:8080").rstrip("/") LANGFUSE_HOST = (os.getenv("LANGFUSE_HOST") or "http://127.0.0.1:3001").rstrip("/")🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@router/main.py` around lines 21 - 29, The module imports `urlparse` twice, so remove the redundant `from urllib.parse import urlparse` and keep only one import near the top of `router/main.py`. Make sure the remaining import is sufficient for any `urlparse` usage elsewhere in the file and leave the other imports untouched.Source: Linters/SAST tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@router/main.py`:
- Around line 3141-3161: The local development fallback in the URL-building
logic is using hardcoded http:// URLs with external_host, which can produce
cleartext links to a non-local host when PUBLIC_BASE_URL is set without matching
ROUTING_DOMAIN. Update the fallback path in the URL construction branch to
either force localhost for the generated URLs or emit a warning when
external_host is not localhost, and keep the behavior consistent in the
parsed_lf, parsed_ll, and parsed_lm URL assembly.
---
Outside diff comments:
In `@litellm/config.yaml`:
- Line 9: The comment for local-qwen-3.6 (35B) is stale and contradicts the
current configuration. Update or remove the note near the top of config.yaml so
it reflects that the model is re-enabled, and make sure the wording matches the
active fallback chains and model_list entries in the same file.
---
Nitpick comments:
In `@litellm/config.yaml`:
- Around line 93-99: The config entry under the local-qwen-3.6 model in
litellm_params uses a hardcoded api_key, so update that block to avoid
committing a plaintext credential. Replace the literal token with an
environment-sourced value or a clearly documented placeholder for the local
llama.cpp setup, and keep the change localized to the existing local-qwen-3.6
configuration.
In `@router/main.py`:
- Around line 21-29: The module imports `urlparse` twice, so remove the
redundant `from urllib.parse import urlparse` and keep only one import near the
top of `router/main.py`. Make sure the remaining import is sufficient for any
`urlparse` usage elsewhere in the file and leave the other imports untouched.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 5ea7333e-e10d-4d4a-8ab5-e882c7f0f71f
📒 Files selected for processing (8)
.agents/AGENTS.mdREADME.mdlitellm/config.yamlpod.yamlrouter/agy_proxy.pyrouter/free_models_roster.jsonrouter/main.pystart-stack.sh
| else: | ||
| # Local development fallback: derive ports and paths dynamically from configuration constants | ||
| parsed_lf = urlparse(LANGFUSE_HOST) | ||
| parsed_ll = urlparse(LITELLM_URL) | ||
| parsed_lm = urlparse(LLAMA_SERVER_URL) | ||
|
|
||
| lf_port = f":{parsed_lf.port}" if parsed_lf.port else "" | ||
| ll_port = f":{parsed_ll.port}" if parsed_ll.port else "" | ||
| lm_port = f":{parsed_lm.port}" if parsed_lm.port else "" | ||
|
|
||
| lf_path = parsed_lf.path | ||
| ll_path = parsed_ll.path or "/ui" | ||
| if not ll_path.endswith("/ui") and not ll_path.endswith("/ui/"): | ||
| ll_path = ll_path.rstrip("/") + "/ui" | ||
| lm_path = parsed_lm.path | ||
|
|
||
| return ( | ||
| f"http://{external_host}{lf_port}{lf_path}", | ||
| f"http://{external_host}{ll_port}{ll_path}", | ||
| f"http://{external_host}{lm_port}{lm_path}" | ||
| ) |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
Local dev fallback uses http:// with potentially non-localhost external_host.
The else branch (local development fallback) hardcodes http:// and uses external_host, which may not be localhost when PUBLIC_BASE_URL is set but doesn't match ROUTING_DOMAIN. In that misconfiguration scenario, the function produces cleartext URLs to a remote host (e.g., http://some-host:3001) instead of either falling back to localhost or raising a clear error.
Consider either:
- Using
"localhost"explicitly in the fallback branch instead ofexternal_host, or - Logging a warning when falling through to the local dev branch with a non-localhost
external_host.
🔒 Suggested improvement: warn on non-localhost fallback
else:
# Local development fallback: derive ports and paths dynamically from configuration constants
+ if external_host not in ("localhost", "127.0.0.1"):
+ logger.warning(
+ "Domain validation failed for external_host %r (ROUTING_DOMAIN=%r); "
+ "falling back to local dev URLs with http://",
+ external_host, domain,
+ )
parsed_lf = urlparse(LANGFUSE_HOST)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| else: | |
| # Local development fallback: derive ports and paths dynamically from configuration constants | |
| parsed_lf = urlparse(LANGFUSE_HOST) | |
| parsed_ll = urlparse(LITELLM_URL) | |
| parsed_lm = urlparse(LLAMA_SERVER_URL) | |
| lf_port = f":{parsed_lf.port}" if parsed_lf.port else "" | |
| ll_port = f":{parsed_ll.port}" if parsed_ll.port else "" | |
| lm_port = f":{parsed_lm.port}" if parsed_lm.port else "" | |
| lf_path = parsed_lf.path | |
| ll_path = parsed_ll.path or "/ui" | |
| if not ll_path.endswith("/ui") and not ll_path.endswith("/ui/"): | |
| ll_path = ll_path.rstrip("/") + "/ui" | |
| lm_path = parsed_lm.path | |
| return ( | |
| f"http://{external_host}{lf_port}{lf_path}", | |
| f"http://{external_host}{ll_port}{ll_path}", | |
| f"http://{external_host}{lm_port}{lm_path}" | |
| ) | |
| else: | |
| # Local development fallback: derive ports and paths dynamically from configuration constants | |
| if external_host not in ("localhost", "127.0.0.1"): | |
| logger.warning( | |
| "Domain validation failed for external_host %r (ROUTING_DOMAIN=%r); " | |
| "falling back to local dev URLs with http://", | |
| external_host, domain, | |
| ) | |
| parsed_lf = urlparse(LANGFUSE_HOST) | |
| parsed_ll = urlparse(LITELLM_URL) | |
| parsed_lm = urlparse(LLAMA_SERVER_URL) | |
| lf_port = f":{parsed_lf.port}" if parsed_lf.port else "" | |
| ll_port = f":{parsed_ll.port}" if parsed_ll.port else "" | |
| lm_port = f":{parsed_lm.port}" if parsed_lm.port else "" | |
| lf_path = parsed_lf.path | |
| ll_path = parsed_ll.path or "/ui" | |
| if not ll_path.endswith("/ui") and not ll_path.endswith("/ui/"): | |
| ll_path = ll_path.rstrip("/") + "/ui" | |
| lm_path = parsed_lm.path | |
| return ( | |
| f"http://{external_host}{lf_port}{lf_path}", | |
| f"http://{external_host}{ll_port}{ll_path}", | |
| f"http://{external_host}{lm_port}{lm_path}" | |
| ) |
🧰 Tools
🪛 ast-grep (0.44.1)
[warning] 3157-3157: Do not make http calls without encryption
Context: f"http://{external_host}{lf_port}{lf_path}"
Note: [CWE-319] Cleartext Transmission of Sensitive Information.
(requests-http)
[warning] 3158-3158: Do not make http calls without encryption
Context: f"http://{external_host}{ll_port}{ll_path}"
Note: [CWE-319] Cleartext Transmission of Sensitive Information.
(requests-http)
[warning] 3159-3159: Do not make http calls without encryption
Context: f"http://{external_host}{lm_port}{lm_path}"
Note: [CWE-319] Cleartext Transmission of Sensitive Information.
(requests-http)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@router/main.py` around lines 3141 - 3161, The local development fallback in
the URL-building logic is using hardcoded http:// URLs with external_host, which
can produce cleartext links to a non-local host when PUBLIC_BASE_URL is set
without matching ROUTING_DOMAIN. Update the fallback path in the URL
construction branch to either force localhost for the generated URLs or emit a
warning when external_host is not localhost, and keep the behavior consistent in
the parsed_lf, parsed_ll, and parsed_lm URL assembly.
Source: Linters/SAST tools
…idelines chore: robust URL resolution, consolidated domain defaults, and generalized runbook (PR #244 fixes)
This PR resolves review comments from PR #243, introduces centralized URL/domain resolution, and refactors base URL calculations.
Key Changes
get_dashboardinto a small, testable utility functionresolve_external_urls(request: Request) -> tuple[str, str, str]inrouter/main.py.PUBLIC_BASE_URL(passed directly to thellm-triage-routercontainer) as the primary source of truth, fallbacking toBASEURL/BASE_URL/request.base_urlsecurely to prevent configuration drift between shell scripts and Python./llm-routing/litellm,/llm-routing/langfuse, and/llm-routing/llama/subdirectories depending on request headers/domain.SERVER_ROOT_PATHand addedPROXY_BASE_URLbehind HAProxy.ROUTING_DOMAINenv variable inrouter/main.py(default:vendeuvre.lan).livenessProbeandreadinessProbeto use direct in-pod endpoints (/pingand/health/readiness) rather than the reverse proxy paths./mnt/DATA/boy/...) and user ID (1002) bindings back to genericWORKDIR_PLACEHOLDER,HOME_PLACEHOLDER, andRUN_USER_PLACEHOLDERtemplates, ensuring portability across multiple environments and restoring proper validation.Summary by Sourcery
Centralize public/base URL handling across the router, deployment scripts, and pod configuration to support subdirectory routing and production domains while restoring portability placeholders and updating routing docs.
Bug Fixes:
Enhancements:
Build:
Deployment:
Documentation:
Summary by CodeRabbit
New Features
Bug Fixes
Documentation